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DETAILED ACTION 

1. This office action is in response to application 10/781443, filed February 18, 
2004. Claims 1-22 are pending in the application and have been examined. 

Information Disclosure Statement 

2. The Information Disclosure Statements filed February 18, 2004 and March 20, 
2006 have been considered in the office action. 

Drawings 

3. The drawings are objected to because drawing one, reference to PC 300 is 
labeled 100. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are 
required in reply to the Office action to avoid abandonment of the application. Any 
amended replacement drawing sheet should include all of the figures appearing on the 
immediate prior version of the sheet, even if only One figure is being amended. The 
figure or figure number of an amended drawing should not be labeled as "amended." If 
a drawing figure is to be canceled, the appropriate figure must be removed from the 
replacement sheet, and where necessary, the remaining figures must be renumbered 
and appropriate changes made to the brief description of the several views of the 
drawings for consistency. Additional replacement sheets may be necessary to show the 
renumbering of the remaining figures. Each drawing sheet submitted after the filing date 
of an application must be labeled in the top margin as either "Replacement Sheet" or 
"New Sheet" pursuant to 37 CFR 1.121(d). If the changes are not accepted by the 
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examiner, the applicant will be notified and informed of any required corrective action in 
the next Office action. The objection to the drawings will not be held in abeyance. 

Specification 

4. The disclosure is objected to because of the following informalities: Page 8 line 5, 
processor 20 should be processor 30. Also paragraphs 43 and 44 discussing Figure 5 
do not reference the components of the drawing. 
Appropriate correction is required. 



Claim Rejections - 35 USC § 101 

5. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

6. Claim 21 is rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. Claim 21 attempts to claim a carrier wave. 
However, this is considered non-statutory under 35 U.S.C. 101. Therefore claim 21 is 
rejected. 
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Claim Rejections - 35 USC § 102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

8. Claims 1, 2, 6-10 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Yang (Visualizing spoken discourse). 

9. Consider claim 1 , Yang teaches a method of determining a predictive model for 
discourse functions (the relationship between discourse and prosody are modeled 
graphically, abstract.) comprising the steps of: 

determining a corpus of speech utterances (Our data for this research consists of 
fifteen dialogue segments extracted from a corpus of 2 hours of spontaneous 
conversation; page 1, column 2, line 18.); 

determining a least one discourse function associated with at least one speech 
utterance (The speech data were digitized and annotated for discourse relations, topic 
structure, interruptions, and speaker turns; page 1, column 2, line 18.); 

determining at least one prosodic feature associated with the at least one 
discourse function (These prosodic features are direct results of the immediacy and 
urgency of the interrupter's demand for additional information; section 2.1.1 last 
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paragraph. Sections 2.1 .1 , - 2.4.2 show other examples of how prosodic features 
indicate discourse features.); and 

determining at least one predictive model of discourse functions based on the 
prosodic features and the discourse functions (the relationship between discourse and 
prosody are modeled graphically for a conversation, abstract.). 

10. Consider claim 2, Yang teaches the method of claim 1 , in which the discourse 
functions are determined based on a theory of discourse analysis (The speech data 
were digitized and annotated for discourse relations; page 1 , column 2, line 18. It is 
inherent that in order to annotate discourse relations, and therefore analysis discourse, 
it must be based on some theory of discourse analysis.) 

1 1 . Consider claim 6, Yang teaches the method of claim 1 , in which the prosodic 
features occur in at least one of a location: preceding, within and following the 
associated discourse function (In order to capture the different domains at which 
prosodic patterns are manifested, we analyzed the data at the within-phrase as well as 
the inter-phrase level, page 2 column 2, line 30. By doing this, prosodic features will be 
analyzed before and after as well as during the discourse function.). 

12. Consider claim 7, Yang teaches the method of claim 1 , in which the prosodic 
features are encoded within a prosodic feature vector (The acoustic measurements of 
fO, amplitude and duration were correlated with the specific characteristics; page 2, 
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column 2, line 23. The data would inherently be reference together for each 
measurement, creating a feature vector.). 

1 3. Consider claim 8, Yang teaches the method of claim 7, in which the prosodic 
feature vector is a multimodal feature vector (The acoustic measurements of fO, 
amplitude and duration were correlated with the specific characteristics; page 2, column 
2, line 23. These three acoustical measurements form a multimodal feature vector for 
the purpose of correlation with discourse characteristics.). 

14. Consider claim 9, Yang teaches the method of claim 1 , in which the discourse 
function is an intra- sentential discourse function (section 2.1 .1 shows the prosodies for 
the discourse function of demanding new information, which is an intra-sentential 
function.) 

1 5. Consider claim 1 0, Yang teaches the method of claim 1 , in which the discourse 
function is an intra- sentential discourse function (section 2.2.3 shows the prosodies for 
the discourse function of topic shifting, which is an inter-sentential function.) 

Claim Rejections - 35 USC § 103 

16. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

17. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 



18. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Yang in 
view of Chino (US Patent 5,761 ,637). 

1 9. Consider claim 3, Yang teaches the method of claim 2, but does not specifically 
teach that the theory of discourse analysis is at least one of: the Linguistic Discourse 
Model, the Unified Linguistic Discourse Model, Rhetorical Structure Theory, Discourse 
Structure Theory and Structured Discourse Representation Theory. 

However, In the same field of Discourse analysis, Chino teaches using Discourse 
Structure Theory for discourse analysis (Figure 7 shows a generated discourse 
structure for the speech input, column 6 line 13- column 7 line 28.) 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the discourse structure theory as taught by Chino for doing 



Application/Control Number: 10/781,443 Page 8 

Art Unit: 2626 

discourse analysis as taught by Yang in order to provide a specific method for discourse 
analysis, allowing one to the structure of a conversation to be observed. 

20. Claims 4, 5, 11, 12, and 13-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yang in view of Shriberg et al (Prosody-Based Automatic 
Segmentation of Speech into Sentences and Topics). 

21 . Consider claim 4, Yang teaches the method of claim 1 , but does not specifically 
teach the predictive models are determined based on at least one of: machine learning, 
rules. 

In the same field of speech analysis, Shriberg teaches predictive models 
determined based on at least one of: machine learning, rules (Prosody models are 
generated using HHM and decision trees in order to segment and determine the topic of 
the speech, which are machine learning techniques, abstract line 3.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the machine learning techniques of Shriberg and apply them to 
the mapping of Prosody to Discourse as taught by Yang in order to provide a completely 
automated method for determining the mappings of Prosody to discourse. 

22. Consider claim 5, Shriberg teaches the method of claim 4, in which the machine 
learning based predictive models are determined based on at least one of: statistics, 
decision trees, Naive Bayes (Prosody models are generated using HHM and decision 
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trees in order to segment and determine the topic of the speech, which are machine 
learning techniques, abstract line 3.). 

23. Consider claim 1 1 , Yang teaches a system for determining predictive discourse 
function models (the relationship between discourse and prosody are modeled 
graphically, abstract.) comprising: 

an input/output circuit for retrieving a corpus of at least one speech utterance (the 
speech data was digitized; page 1, column 2, line 18.); 

and determining prosodic features associated with the at least one speech 
utterance (Acoustic measurements of the speech are taken such as fO, amplitude, page 
1, column 2 line 23.), and which determines at least one discourse function associated 
with the corpus of at least one speech utterance (The speech data were digitized and 
annotated for discourse relations, topic structure, interruptions, and speaker turns; page 
1, column 2, line 18.) and determines at least one prosodic feature associated with the 
at least one discourse function (These prosodic features are direct results of the 
immediacy and urgency of the interrupter's demand for additional information; section 
2.1 .1 last paragraph. Sections 2.1 .1 , - 2.4.2 show other examples of how prosodic 
features indicate discourse features.) and determines a predictive model for discourse 
functions based on the prosodic features and the discourse function (the relationship 
between discourse and prosody are modeled graphically for a conversation, abstract.). 

Yang does not specifically teach that the processor itself does the analysis to 
determine the model between prosody and discourse. 
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In the same field of text analysis, Shriberg teaches using the processor to 
relate prosody to text functions. (Prosody models are generated using HHM and 
decision trees in order to segment and determine the topic of the speech, which are 
machine learning techniques, making a processor inherent, abstract line 3.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the machine learning techniques with a processor of Shriberg 
and apply them to the mapping of Prosody to Discourse as taught by Yang in order to 
provide a completely automated method for determining the mappings of Prosody to 
discourse. 

24. Consider claim 12, Yang teaches the system of claim 1 1 , in which the discourse 
functions are determined based on a theory of discourse analysis (The speech data 
were digitized and annotated for discourse relations; page 1, column 2, line 18. It is 
inherent that in order to annotate discourse relations, and therefore analysis discourse, 
it must be based on some theory of discourse analysis.) 

25. Consider claim 14, Shriberg teaches the system of claim 11, wherein the 
predictive models are determined based on at least one of: machine learning, rules 
(Prosody models are generated using HHM and decision trees in order to segment and 
determine the topic of the speech, which are machine learning techniques, abstract line 

3.). 
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26. Consider claim 15, Shriberg teaches the system of claim 14, in. which the 
machine learning based predictive models are determined based on at least one of: 
statistics, decision trees, Naive Bayes (Prosody models are generated using HHM and 
decision trees in order to segment and determine the topic of the speech, which are 
machine learning techniques, abstract line 3.). 

27. Consider claim 16, Yang teaches the system of claim 1 1 , in which the prosodic 
features occur in at least one of a location: preceding, within and following the 
associated discourse function (In order to capture the different domains at which 
prosodic patterns are manifested, we analyzed the data at the within-phrase as well as 
the inter-phrase level, page 2 column 2, line 30. By doing this, prosodic features will be 
analyzed before and after as well as during the discourse function.). 

28. Consider claim 17, Yang teaches the system of claim 1 1 , in which the prosodic 
features are encoded within a prosodic feature vector (The acoustic measurements of 
fO, amplitude and duration were correlated with the specific characteristics; page 2, 
column 2, line 23. The data would inherently be reference together for each 
measurement, creating a feature vector.). 

29. Consider claim 1 8, Yang teaches the system of claim 1 7, in which the prosodic 
feature vector is a multimodal feature vector (The acoustic measurements of fO, 
amplitude and duration were correlated with the specific characteristics; page 2, column 
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2, line 23. These three acoustical measurements form a multimodal feature vector for 
the purpose of correlation with discourse characteristics.). 

30. Consider claim 19, Yang teaches the system of claim 1 1 , in which the discourse 
function is an intra- sentential discourse function (section 2.1.1 shows the prosodies for 
the discourse function of demanding new information, which is an intra-sentential 
function.) 

31 . Consider claim 20, Yang teaches the system of claim 1 1 , in which the discourse 
function is an intra- sentential discourse function (section 2.2.3 shows the prosodies for 
the discourse function of topic shifting, which is an inter-sentential function.) 

32. Consider claim 21 , Yang teaches: 

determining a corpus of speech utterances (Our data for this research consists of 
fifteen dialogue segments extracted from a corpus of 2 hours of spontaneous 
conversation; page 1, column 2, line 18.); 

determining a least one discourse function associated with at least one speech 
utterance (The speech data were digitized and annotated for discourse relations, topic 
structure, interruptions, and speaker turns; page 1, column 2, line 18.); 

determining at least one prosodic feature associated with the at least one 
discourse function (sections 2. 1 . 1 , - 2.4.2 discuss how prosodic features indicate 
discourse features, such as demanding of information, etc.); and 
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determining at least one predictive model of discourse functions based on the 
prosodic features and the discourse functions (the relationship between discourse and 
prosody are modeled graphically for a conversation, abstract.). 

But Yang does not specifically teach a carrier wave encoded to transmit a control 
program with instructions to complete the above steps. 

In the same field of speech analysis, Shriberg teaches using a computer program 
to do speech analysis (Prosody models are generated using HHM and decision trees in 
order to segment and determine the topic of the speech, which are machine learning 
techniques, making a software program inherent, abstract line 3. A software program 
can obviously be carried on a carrier wave.). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the machine learning techniques with a processor of Shriberg 
and apply them to the mapping of Prosody to Discourse as taught by Yang in order to 
provide a completely automated method for determining the mappings of Prosody to 
discourse. 

Consider claim 22, Yang teaches: 

determining a corpus of speech utterances (Our data for this research consists 
of fifteen dialogue segments extracted from a corpus of 2 hours of spontaneous 
conversation; page 1, column 2, line 18.); 
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determining a least one discourse function associated with at least one speech 
utterance (The speech data were digitized and annotated for discourse relations, topic 
structure, interruptions, and speaker turns; page 1, column 2, line 18.); 

determining at least one prosodic feature associated with the at least one 
discourse function (These prosodic features are direct results of the immediacy and 
urgency of the interrupter's demand for additional information; section 2.1 .1 last 
paragraph. Sections 2.1 .1 ; - 2.4.2 show other examples of how prosodic features 
indicate discourse features.); and 

determining at least one predictive model of discourse functions based on the 
prosodic features and the discourse functions (the relationship between discourse and 
prosody are modeled graphically for a conversation, abstract.). 

But Yang does not specifically teach that these steps are executed with a 
computer readable medium comprising computer code usable to complete the above 
steps. 

In the same field of speech analysis, Shriberg teaches a computer readable 
medium comprising computer code usable to complete speech analysis (Prosody 
models are generated using HHM and decision trees in order to segment and determine 
the topic of the speech, which are machine learning techniques, making a software 
program inherent, abstract line 3. A software program must inherently be stored on a 
computer readable medium in order to be executed ). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the machine learning techniques with a processor of Shriberg 
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and apply them to the mapping of Prosody to Discourse as taught by Yang in order to 
provide a completely automated method for determining the mappings of Prosody to 
discourse. 

33. Claim 1 3 is rejected under 35 U.S. C. 103(a) as being unpatentable oyer Yang in 
view of Shriberg as applied to claim 1 1 above, and further in view of Chino. 

34. Consider claim 13, Yang in view of Shriberg teaches the system of claim 12, but 
does not specifically teach that the theory of discourse analysis is at least one of: the 
Linguistic Discourse Model, the Unified Linguistic Discourse Model, Rhetorical Structure 
Theory, Discourse Structure Theory and Structured Discourse Representation Theory. 

However, In the same field of Discourse analysis, Chino teaches using Discourse 
Structure Theory for discourse analysis (Figure 7 shows a generated discourse 
structure for the speech input, column 6 line 13- column 7 line 28.) 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
of the invention to use the discourse structure theory as taught by Chino for doing 
discourse analysis as taught by Yang in view of Shriberg in order to provide a specific 
method for discourse analysis, allowing one to the structure of a conversation to be 
observed. 
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Conclusion 



35. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure is listed on the notice of references cited. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Douglas C. Godbold whose telephone number is (571) 
270-1451 . The examiner can normally be reached on Monday-Thursday 7:00am- 
4:30pm Friday 7:00am-3:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Patrick Edouard can be reached on (571) 272-7603. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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PATRICK N. EDOUARD 
SUPERVISORY PATENT EXAMINER 




